Numerical Results for Ground States of Mean-Field Spin Glasses at low Connectivities 
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An extensive list of results for the ground state properties of spin glasses on random graphs is 
presented. These results provide a timely benchmark for currently developing theoretical techniques 
based on replica symmetry breaking that are being tested on mean-field models at low connectivity. 
Comparison with existing replica results for such models verifies the strength of those techniques. 
Yet, we find that spin glasses on fixed-connectivity graphs (Bethe lattices) exhibit a richer phe- 
nomenology than has been anticipated by theory. Our data prove to be sufficiently accurate to 
speculate about some exact results. 
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A theoretical understanding of the intricate dynamics 
of disordered systems has been a major goal of statistical 
physics at least since the introduction of the Edwards- 
Anderson spin glass model . Already the study of the 
equilibrium at low temperature, a state real disordered 
materials rarely achieve [^J, reveals a stunning range of 
new phenomena, even in the simplest models such as the 
Sherrington and Kirkpatrick model (SK) where all spins 
are mutually connected |j, ^, g, [6J. As an intermediate 
step in extending the mean-field techniques toward finite- 
dimensional models, spin glasses on random graphs are 
an area of active research. Those systems have been of in- 
terest from early on because they combine infinite-range 
connections (like SK) with a finite, decidedly low con- 
nectivity. But those earlier studies have focused either 
on temperatures at the glass transition Q, on purely 
replica symmetric (RS) solutions or on perturba- 
tive approaches in the (SK-)limit of large connectivity 

Simultaneously, the formal similarity between spin- 
glass Hamiltonians and the objective function of com- 
binatorial optimization problems has been realized JlJ] 
and exploited to make RS predictions, for instance, for 
the bipartitioning problem on random graphs fl3| , [ljL fl5| . 
This connection, with the discovery of phase transitions 
in combinatorial optimization problems [l6| and the ap- 
plication of replica techniques to their study |]l7f , has 
recently rejuvenated interest in spin glasses on random 
graphs. But to obtain quantitatively valuable predictions 
for NP-hard problems required the application of replica 
symmetry breaking (RSB) Q to those problems at finite 
connectivities and low temperatures which was accom- 
plished recently [l8[ ||, §0|. Finally, these RSB methods 
are now being applied to spin glasses on random graphs, 
producing quantitatively valuable results |ll| |l| at ac- 
curacies below 0.1%. At this level of accuracy, a compar- 
ison between theoretical and simulation results becomes 
valuable at least in two respects: Convergence of the nu- 
merical with the RSB result can verify the assumptions 
underlying RSB as well as the quality of the numerical 
method used to approximate an NP-hard problem. 



In this letter we apply the extremal optimization (EO) 
heuristic |2^, ^3) to investigate the ground state proper- 
ties of spin glasses on random graphs. With this method 
we have sampled system sizes up to n = 4096 on low- 
connectivity graphs. We have obtained high-accuracy 
results for the ground-state energies of spin glasses on 
ordinary random graphs (ORG) with fluctuating connec- 
tivities, and for Bethe-lattice graphs (BL) with fixed con- 
nectivities. On a smaller sample of BL, we have also 
obtained results for the entropy of such graphs. The en- 
ergies are in excellent agreement with RSB predictions 
for low-connectivity BL pi[| . Both, the energies and en- 
tropies reveal a sensitivity to the even-oddness of the BL, 
which may explain inconsistencies with results in the SK 
limit No such inconsistency arises for ORG and the 
numerical extrapolation is in good agreement with ana- 
lytic results for the large-connectivity limit in RSB ||. 
The BL energies seem to fall (within 0.2%) on a sim- 
ple line ranging from the 2-connected graphs to the SK- 
limit, but only for even integer connectivities, without 
any obvious interpolation. The BL entropies decrease lin- 
early with the inverse connectivity for odd connectivities, 
but are already consistent with zero (within accuracy) at 
small even connectivity. More details of the numerical 
procedure is given elsewhere |24|, [25] . 

Of the two types of random graphs are considered for 
this study, the BL are regular random graphs |^6| . These 
graphs consist of n vertices where each vertex possesses a 
fixed number k+ 1 of bonds ]r| [l^, ^l) with randomly se- 
lected other vertices. Alternatively, ORG are obtained by 
randomly connecting any pair of vertices with a specified 
probability p = c/(n— 1), leading to a graph of average 
connectivity c but where the connectivities of individual 
vertices are Poissonian distributed p6| . Note that each 
vertex' connectivity, and thus fc + 1, is inherently discrete, 
while c can take on any real value. 

Once a graph of connectivity c is generated, ran- 
domly chosen quenched couplings Jij G { — 1,+1} are 
assigned to existing bonds between neighboring vertices 
i and j. Each vertex i is occupied by a spin variable 
Xi e { — !,+!}. The energy of the system is defined as 
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the difference in number between violated bonds and sat- 
isfied bonds, H = — E{ bonds} Ji,j x i x j> an d we will focus 
on the energy and entropy per spin, resp., 
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lnfi, 



(1) 



where fi is the degeneracy of the configurations exhibit- 
ing the ground state energy. 

For our numerical procedure we used the following im- 
plementation of EO (2^, |2^]: For a given spin config- 
uration on a graph, assign to each spin xi a "fitness" 
Ai = — ^violated bonds = — 0, — f , —2, . . . , — a, so that 
e c = —J2i^i/(^ n ) i s satisfied. Here, a is the integer 
connectivity of vertex i, and Cj = k + f for every vertex 
in BL. If c max = maxi c,, each spin falls into one of only 
Cmax + 1 possible states. Say, currently there are n c 



spins with the worst fitness, A 



with 



A = -Cm, 
fitness A 



1, and so on up to no spins with the best 



0, where 



n. Now draw a "rank" I 



according to the distribution P(l) ~ I T . Then, deter- 



mine < j < c max such that J2i= 



Finally select any one of the nj spins in state j and re- 
verse its orientation unconditionally. As a result, it and 
its neighboring spins change their fitness. After all the 
effected A's and n's are reevaluated, a new spin is chosen 
for an update. 

The arguments given in and a few experiments in- 
dicate that t = 1 .3 is a satisfactory choice to find ground 
states efficiently on either type of graph. Our imple- 
mentation restarts for each instance at least r max = 4 
times with new random initial spin assignments, exe- 
cuting « 0.1 n 3 updates per run. If a new, lower-than- 
previous energy state is encountered in run r, we adjust 
''max = 2 + 2r for that instance so that EO runs at least 
twice as many restarts as were necessary to find the low- 
est state in the first place. Especially for small n, r max 
hardly ever exceeds 4; for larger n & few graphs require 
up to 30 restarts before termination. Since EO perpet- 
ually explores new configurations it is well suited to ex- 
plore also the degeneracy O of low-energy states. In these 
runs, we used a similar approach to the above, except for 
setting r max = 8 + 2r, where r is the latest run in which 
a new configuration of the lowest energy was located. 

We have simulated spin glasses on BL with this algo- 
rithm for k + 1 between 3 and 26, and graph sizes n = 2 l 
for I = 5, 6, . . . ,10 to obtain results for ground state en- 
ergies pH - In particular, for k + 1 = 3 we have used 
the methods described in Ref. |2f| to reach system sizes 
of n = 4096. In a separate simulation, using r = 1.4, 
we have explored BL of size n £ [16 . . . 256] to deter- 
mine their entropy. We have used the same algorithm, 
preceded by a graph reduction procedure p5[ , to study 
ORG ranging from n < 2 15 for c = 2 to n < Ir at c = 25. 
Amazingly, as is shown in Refs. |24|, p5|, in all these cases 
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FIG. 1: Extrapolation plot for the BL energies (above) and 
entropies (below) for k + 1 = 3 obtained with EO for finite n 
[ pif . The data is plotted vs. 1/n 2 / 3 and is fitted according 
to Eq. (Q). (Data points were weighted with respect to n and 
the inverse of the error.) For n — > 00 the extrapolation for 
the energy gives a = —1.2716(1), way above the RS result 
but consistent with the 1RSB result fell], both indicated by 
horizontal lines. The extrapolation for the entropy gives S3 = 
0.0102(10). 



our data can be extrapolated for n 

A 



00 via 



,2/3 



(n — > 00). 



(2) 



Deviations from these scaling corrections are generally 
small |24j and we assume Eq. @ to be exact here. 
There does not appear to be a theoretical justification 
for Eq. (0), but in Fig. |f| we show representatively the 
large-n extrapolation of our data for k + 1 = 3. Both, 
finite-n energies and entropies, appear to scale linearly 
when plotted for f/n 2 / 3 . The extrapolation results for 
the energies of BL and ORG are given in Tab. ||. The 
extrapolation results for the BL entropies are given in 
Tab. 0. Variation in the (estimated) errors reflect dif- 
ferences in computational effort (number of instances, 
largest n) and in the quality of the extrapolation p4j . 

We can compare our results with existing theoretical 
predictions at the RS and the 1RSB level at least for 
the case of BL at k + 1 = 3. For this case, Ref. pjj ] 
reproduced = —1.2777 at the RS level, and yielded 
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TABLE I: Extrapolated energies per spin for BL (left) and 
ORG (right). Although only integer values of the average 
connectivity c were considered, it can take on any real value, 
unlike k + 1. 
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-1.2716(1) 
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TABLE II: Extrapolated entropies per spin for BL. 
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es = —1.2717 at the 1RSB level (further replica correc- 
tions are expected to be small). These values and our 
extrapolation result of = —1.2716(1) are indicated in 
Fig. [l]. Clearly, the extrapolation result is extremely close 
to the 1RSB results, but inconsistent with the RS result. 

We have also used EO to sample the degeneracy il of 
the lowest-energy states found for BL. In these simula- 
tions we focused on smaller system sizes of n < 256 for 
k + 1 = 3, . . . , 9 and 10, 14, . . . ,26 only. As a test for the 
accuracy of our implementation, we have run the simu- 
lation for k + 1 = 3 a second time on identical instances, 
using different initial conditions and n/5 more updates, 
obtaining identical results for each instance. 

When plotted for l/(k + 1) in Fig. |, the BL en- 
tropies for even connectivities decrease about linearly to- 
ward zero. The entropies for odd connectivities, clearly 
non-zero at k + 1 = 3 (see Fig. Q), drop more rapidly 
and are essentially indistinguishable from zero already at 
k + 1 = 9. While any rapid, smooth decay could easily 
escape our limited accuracy, the plot still raises the ques- 
tion whether there may be a finite connectivity beyond 
which odd entropies vanish. The qualitative difference 
between even and odd BL entropies can be understood 
in the presents or absence, resp., of "free spins," a finite 
fraction of spins in the ground state which violate exactly 
half of their bonds and may flip at no cost |2^] . 

We have also plotted all BL and ORG energies as 
e c j\fc vs. 1/c (where c = k + 1 for BL) in Fig. ||. We 
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FIG. 2: Asymptotic plot of the extrapolated entropies from 
Tab . [n| as a function of 1 / ( + 1 ) . The data for even k+1 seems 
to vanish linearly with l/(k + 1) (dashed line). The data for 
odd k + 1 drops more precipitously, and can not reasonably 
be fitted at this level of accuracy. 



expect that lim c ^ oo e c /y/c = Esk = —0.76321 for RSB 
H ^8). All energies for ORG appear to fall on a single 
smooth curve. We can fit those energies with a parabola, 
projecting Eg]^ G —0.761. In contrast, the BL energies 
split into even and odd values, each located apparently on 
a simple line. Each line separately extrapolates very close 
to the exact value: Efg 1 » -0.763 and E° d g » -0.765. 
Amazingly, the trivial value of 62 = — 1 is very close to 
the linear fit for the even results. Clearly, a function 
that would interpolate continuously all the BL energies 
will have to be very complicated (oscillatory). But we 
may speculate that its envelope for the even e k +i is a 
simple line, passing e2 = — 1 and the SK result: 



£ k+ i = E S K^/k + 1 



2E SK + V2 



Vk + T 



(3) 



In Fig. ^ we plot the deviation e = ek+i/£k+i — 1 of 
the extrapolated energies from Eq. fj) for even k + 1. 
While the extrapolated values do not fall exactly onto 
the proposed function, they are all within about 0.2% of 
it. In fact, all points are slightly too high, which may 
indicate a more complex functional correction to Eq. (|^) , 
or a systematic error, say, in the extrapolation due to 
higher-order corrections. 

It has been pointed out [^J that Eq. (S) would imply 
that a first-order perturbation around the trivial k+1 = 2 
solution would be exact and give the RSB result for the 
SK model. But the obvious continuation of BL off the 
even integers fails to interpolate the data smoothly. If 
we "interpolate" BL for each k + 1 = 2,4, 6, . . . with a 
mix of (1 — p)n {k + l)-vertices and pn (k + 3)-vertices for 
< p < 1, the resulting energies only provide a set of se- 
cants, ek+i+2 P = pek+3 + (1 — p)ek+i, to the even-integer 
data. We have also plotted our (somewhat less accu- 
rate) extrapolation results for those interpolating graphs 
in Fig. |3| On this scale, the singular behavior of this 
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FIG. 3: Plot of the rescaled extrapolated energies, e c /^/c, as 
a function of 1/c for ORG (squares) and BL (circles), where 
c = k + 1. The BL data appears to fall on two separate 
straight lines for even and for odd k + 1, including the trivial 
result, e2 = — 1 (diamond). In all cases, the fits (dashed 
lines) provide an reasonable estimate for Esk = —0.76321 
(horizontal line) at infinite connectivity. 
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FIG. 4: Plot of the deviation e of the BL energies for even 
k + 1 (circles) relative to Eq. (^) as a function of l/(k + 1). 
All BL data deviates a£ worst by 0.2%. The point at fe + 1 = 2 
(diamond) is exact. Energies from the interpolating graphs 
(crosses) do not smoothly interpolate the BL data. Dashed 
lines are derived from the secants ek+i+2 P = pet+a + (1 — 
p)efc+i, k + 1 — 2,4 and < p < 1, and clearly trace the 
interpolating data. 



continuation at the even integers becomes obvious. 

We can further compare our extrapolated energies with 
perturbative calculations in the SK-limit of infinite con- 
nectivity H [h], A recent RSB calculation indi- 
cates a 1 /c-correction for the ground state energy of 
ff L fa -0.32 for BL and f? RG « 0.17 for ORG (see 
Figs. 3 and 4 of Ref. §). While our (crude) fit in Fig. | 
predicts a slope at the origin of w 0.16 for ORG, the 
slopes for cither even or odd BL data would predict 
w 0.11 - 0.10, or 0.1122 from Eq. (g), far from the per- 
turbative result. It appears that the oscillation between 
even and odd k + 1 complicates also the analytic contin- 
uation of the BL problem for large connectivities. 



It will be most interesting to see how well upcoming 
RSB calculations at T = for even k + 1 will correspond 
to the proposed function in Eq. (||) , or to the extrapolated 
energies in Tabs. | in general. While the EO algorithm 
in itself can not provide information about the physics at 
T > 0, the results presented in this Letter are sufficiently 
promising to apply EO also to sample other models |23[ ] 
and more complicated properties of the ground states, 
such as overlap distributions and excitations. 



[1 

[2 

[3 

[4 

[5 
[6 

ir 

[8 

[9 
[10 

[11 

[12 

[13 
[14 

[is; 

[16 
[17 



[18 
[19 

[20 



[21 
[22 

[23 

[24 
[25 

[26 

[27 



Electronic address: www.physics.emory.edu/faculty/boettchei 
S. F. Edwards and P. W. Anderson, J. Phys. F: Metal 
Phys. 5, 965-974 (1975). 

K. H. Fischer and J. A. Hertz, Spin Glasses (Cambridge, 
1991). 

D. Sherrington and S. Kirkpatrick, Phys. Rev. Lett. 35, 
1792 (1975). 

D. J. Thouless, P. W. Anderson, and R. G. Palmer, Phil. 
Mag. 35, 593-601 (1977). 

G. Parisi, J. Phys. A: Math. Gen. 13, L115-121 (1980). 
M. Mezard, G. Parisi, and M. A. Virasoro, Spin Glass 
Theory and Beyond, (World Scientific, Singapore, 1987). 
L. Viana and A. J. Bray, J. Phys. C 18, 3037 (1985). 
I. Kanter and H. Sompolinsky, Phys. Rev. Lett. 58, 164 
(1987). 



G. Parisi and F. Tria, cond-mat/0207144 



C. De Dominicis and Y. Y. Goldschmidt, J. Phys. A: 
Math. Gen. 22, L775 (1989). 

P.-Y. Lai and Y. Y. Goldschmidt, J. Phys. A: Math. Gen. 
23, 3329 (1990). 

S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, Science 
220, 671-680 (1983). 

M. Mezard and G. Parisi, Europhys. Lett. 3, 1067 (1987). 
K. Y. M. Wong and D. Sherrington, J. Phys. A: Math. 
Gen. 20, L793 (1987). 

J. R. Banavar, D. Sherrington, and N. Sourlas, J. Phys. 
A: Math. Gen 20, LI (1987). 

B. A. Huberman and T. Hogg, Artificial Intelligence 33, 
155-171 (1987). 

R. Monasson, R. Zecchina, S. Kirkpatrick, B. Selman, 
and L. Troyansky, Nature 400, 133-137 (1999), and Ran- 
dom Struct. Alg. 15, 414-435 (1999). 
M. Mezard and G. Parisi, Eur. Phys. J. B 20, 217 (2001). 
S. Franz, M. Leone, F. Ricci-Tersenghi, and R. Zecchina 
R, Phys. Rev. Lett. 87, 7209-7212 (2001). 
M. Mezard, G. Parisi and R. Zecchina, Sciencexpress, 
June 2002, see 



http://www.sciencemag.org/scicnccxDicss/rccent.shtm] 



M. Mezard and G. Parisi, |sond-mat/0207121 
S. Boettcher and A. G. Percus, Artificial Intelligence 119 
275-286 (2000). 

S. Boettcher and A. G. Percus, Phys. Rev. Lett. 86, 5211- 
5214 (2001) 



S. Boettcher, |cond-mat/020819e . 

S. Boettcher, Numerical Results for Ground States of 

Spin Glasses on Random Graphs, in preparation. 

B. Bollobas, Random Graphs, (Academic Press, London, 

1985). 

S. Boettcher and M. Grigni, J. Phys. A: Math. Gen. 35 



■5 



1109-1123 (2002). [29] M. Mezard, private communication. 

A. Crisanti, T. Rizzo, Phys. Rev. E 65, 046137 (2002). 



